This specification relates to information retrieval and image classification.
The Internet provides access to a wide variety of resources, such as image files, audio files, video files, and web pages. A search system can identify resources in response to a text query that includes one or more search terms or phrases. The search system ranks the resources based on their relevance to the search query and provides search results that link to the identified resources. The search results are typically ordered for viewing according to the rank.
Images are among the resources the search system identifies. The search system can utilize an image relevance model to evaluate the relevance of queries to images. For example, the relevance of an image to a user query can be determined, in part, based on textual content depicted in the image or textual content associated with the image, e.g., textual content appearing on a web page in association with the image. The relevance of an image to a query can also be determined by analyzing the way in which users who submit user queries interact with images responsive to the user queries. For example, an image search system may respond to a user query with one or more images, which are then displayed on a user device. If the user clicks on or otherwise interacts with one or more images, then this user interaction can be considered an indication that those images were good (relevant) responses to the user query.